Method for determining movement patterns during a dental treatment

ABSTRACT

A method includes selecting a treatment mode displayed on a screen of a mobile phone or tablet PC, using a camera of the tablet PC or mobile phone to capture an image area containing (i) one or both pupils, the nose, eyebrows, eye hole, lips, ears, cheeks and/or chin of the person, and (ii) a toothbrush moving at least in an X-/Y-plane, and wirelessly receiving sensor data from the toothbrush in the mobile phone or tablet PC. The sensor data contains at least rotational movements and/or acceleration of the toothbrush captured by an acceleration sensor and/or a rotation sensor, disposed in or on the toothbrush. Thereafter, the method includes processing the sensor data in a processor, thereby generating processed sensor data, generating correction information based on the processed sensor data and the selected treatment mode, and displaying the correction information on the screen of the mobile phone or tablet PC.

CROSS-REFERENCE

This application is a continuation of U.S. patent application Ser. No.16/210,111, now pending, which is a continuation of U.S. patentapplication Ser. No. 14/898,176, now U.S. Pat. No. 10,172,552, which isthe U.S. national stage of International Patent Application No.PCT/EP2014/062077 filed on Jun. 11, 2014, which claims priority toGerman Patent Application No. 10 2013 010 292.8 filed on Jun. 19, 2013,German Patent Application No. 10 2013 015 537.1 filed on Sep. 18, 2013and German Patent Application No. 10 2013 021 492.0 filed on Dec. 17,2013.

TECHNICAL FIELD

The invention generally relates to systems and methods for determiningmovement patterns during a dental treatment, such as a teeth cleaning.

RELATED ART

Due to relatively large intervals of time (e.g., several months or evenyears) between dental visits, a problem results in that the instructionsof the dentist with respect to frequency, duration and manner oftreatment (e.g., cleaning) are easily forgotten, or over timenon-optimal motion sequences (e.g., brushing movements or brush strokes)establish (become habitual), whereby oral hygiene is not optimallyensured and/or deterioration of the oral cavity, in particular of solidand/or soft oral parts, like e.g. the teeth, the jaw and/or the gum, iscaused.

To address this problem, European patent publication EP 1 379 149 B1discloses a method for monitoring the position of a toothbrush withrespect to the teeth of a person, which comprises: providing atoothbrush with a first position sensor, wherein the first positionsensor sensitively reacts to changes of the position and orientation ofthe toothbrush, providing a second position sensor in a fixed positionalrelationship with respect to the teeth, wherein the second positionsensor sensitively reacts to changes of the position and orientation ofthe teeth, and transmitting the outputs of the first position sensor andthe second position sensor to a processing apparatus, so that theprocessing apparatus compares both sensor outputs in order to monitorthe position of the toothbrush with respect to the teeth during thetooth brushing.

However, this known system is disadvantageous, since expensive sensorsmust be attached to the face in a defined manner, which makes the systemcostly and complex, thereby making a high market acceptance marketdifficult.

In the article “Playful Toothbrush: UbiComp Technology for TeachingTooth Brushing to Kindergarten Children”, Yu-Chen Chang et. al. describea system in which the movement of the toothbrush is optically detected.However, this system is disadvantageous, since the detected movementstake place in an undefined manner in space, such that a movement of thehead during the optical detection causes a corruption of the detecteddata. Furthermore, expensive modifications of the toothbrush arerequired, which makes the system complex and costly and thus it appearsto be unlikely to achieve mass market acceptance. Additionally, it isproblematic that the system has to be installed in a fixed manner inspace, so that a camera can detect the modified toothbrush in a definedmanner.

Further, both of the above-described known systems are based on given(predetermined) data sets, such that an enhancement of the knowledgewith respect to tooth brushing movements is not possible.

SUMMARY

Therefore, in one non-limiting aspect of the present teachings, methodsand systems are disclosed, which facilitate an optimization of oralhealth, such as oral hygiene, tooth health, jaw health and/or gumhealth. In addition or in the alternative, the present teachings alsofacilitate a control, such as continuous or permanent control, or amonitoring, such as continuous or permanent monitoring, of the oralhygiene. In addition or in the alternative, the present teachingsprovide simple, low-priced and widely-usable methods and devices.

A method according to another aspect of the present teachings preferablycomprises: (i) moving a dental treatment means at least in an X/Y planein order to treat surface portions of the teeth, (ii) capturing(recording) data relating to at least one movement parameter, such asthe movement direction or rotation, the acceleration, the path and/orthe speed, of the dental treatment means relative to a first referencesystem that moves with the head of the treated person using at least oneoptical detection device, (iii) defining a second three-dimensionalreference system using a processor device based on characteristic bodypoints of the hand, body lines of the hand and/or body surfaces of thehand that is guiding (moving) the dental treatment means, wherein thesecond reference system moves at least at times (preferably constantly)during movement of the dental treatment means in space with the dentaltreatment means, (iv) providing the recorded data to a processor devicefor determination of the movement pattern and (v) determining themovement pattern.

Preferably, the dental treatment means is moved at least at times in athree dimensional manner, i.e. in the X-, Y-, and Z-directions, and themovement parameters are particular preferably dependent on the directionand/or orientation recorded (captured). The X-Y plane extends in theX-direction and in the Y-direction, possible other planes extend in ananalogous manner, and the X-axis, Y-axis and Z-axis are orthogonal toeach other.

Alternatively or additionally, a method of at least partiallydetermining a movement pattern of the dental treatment means (e.g., atoothbrush) during the dental treatment (e.g., a teeth cleaning),preferably in an individual-related (personalized) manner, according toanother aspect of the present teachings preferably comprises at least(i) optical recording (capturing) of data, such as by using at least onecamera, with respect to at least one movement parameter, such as themovement direction or rotation, the acceleration, the traveled pathand/or the speed, of the dental treatment means, (ii) image separationof the dental treatment means, such as of the hand guiding the dentaltreatment means, from at least one further image part recorded(captured) using the optical detection device and (iii) determining atleast one movement parameter corresponding to the movement pattern.

Such embodiments are advantageous, since the data recording (capture)can take place anywhere and at any time and thus a monitoring andobjective evaluation of the dental treatment can take place also at thehomes of private individuals at any time, i.e. in a time-independentmanner. The recorded (captured) data are to be processed, e.g., in realtime, in such a manner, or are preferably processed in real time in sucha manner, that the movement pattern is analyzable by e.g. an attendingdentist or a private individual or a patient, whereby individualized(customized) adaptations (adjustments) of the movement patterns canimmediately be made during a dental treatment. The data collected insuch a manner can be used not only individually for acute (immediate)optimization of the movement pattern, but can also be used in largerscale, e.g., by health insurance companies and/or research institutes.By transferring data collected from multiple users to a centralevaluation station, on the one hand, it opens up the possibility, e.g.,to process the raw data in a centralized manner with adequate computingpower, to compare it with optimal movement schematas and to send it tothe local end-user devices; on the other hand, it is additionally oralternatively possible to provide data for large clinical studies forresearch purposes or for commercial statistical usage (e.g. by healthinsurance companies).

According to a further embodiment of the present teachings, a method fordetermining at least one movement characteristic preferably comprises atleast: (i) carrying out one of the above-described methods multipletimes, such as on different end-user devices for recording (capturing)movement patterns of different persons and preferably the statisticalevaluation thereof, wherein it is additionally or alternatively alsoconceivable that a movement characteristic is generated or determinedfor one person, and (ii) determining at least one coinciding property,such as a movement parameter and/or the dental treatment duration, ofthe determined movement patterns or the recorded data.

This embodiment is advantageous, since one or more movementcharacteristics are derivable from the movement pattern of individualpersons, which movement characteristics are representative of the dentaltreatment behavior of the respective person. However, it is additionallyor alternatively conceivable that the movement patterns or movementcharacteristics of individual persons are to be grouped or are groupedbased on coinciding features. Furthermore, the determined movementpatterns or the recorded data may be correlated with characteristicstandard data, which are provided in a database, wherein the database isat least partially provided (stored) locally on a mobile end-user deviceand/or at least partially on a server.

Furthermore, during usage of the device, it is possible to provide astep of outputting (e.g., optical, haptic and/or acoustic outputting)information concerning correction-, standard- and/or ideal-movements,such as in accordance with the currently-, i.e. e.g. in real time,determined movement patterns or in accordance with an already determinedmovement pattern or a movement characteristic. Optical outputting hereinpreferably means the visual depiction (display) of correction-,standard- and/or ideal-movements on a screen or a projection surface,wherein visual outputting can be understood as the depiction (display)of individual images and/or an animated movie. Acoustic outputtingherein preferably means the outputting of specific tones and/or theoutputting of spoken instructions. Haptic outputting can be understoode.g. as a defined output of vibrations. In particular with respect tothe optical outputting, the correction-, standard-, and/orideal-movements preferably can be output adjacent, over and/or incorrelation to the actually recorded (captured) movement patterns or tothe actually recorded (captured) movement characteristics. Correctionmovements can herein include, e.g., the display of a suggestion for amovement change or adjustment as compared to the recorded (captured)movement pattern. Standard movements can be e.g. the average movementsof a defined group of persons or those movements known from clinicalstudies. Ideal movements can be e.g. movements which are very gentle,cleansing and/or time-efficient, etc.

According to a further embodiment of the present teachings, a method formonitoring dental treatment movements and for outputting correctionparameters for adapting the dental treatment movements preferablycomprises the above-described method of at least partially determining amovement pattern of the dental treatment means (e.g., a toothbrush)during a dental treatment (e.g., a teeth cleaning), and furtherincludes: (i) moving the dental treatment means at least in anX-/Y-plane to treat surface parts of the teeth, wherein data withrespect to at least one movement parameter of the dental treatment means(e.g., the movement direction or rotation, the acceleration, the pathand/or the speed) are recorded (captured) by preferably using an opticaldetection device, such as a camera, and (ii) providing the recorded(captured) data to a processor device or a system for determination ofthe movement pattern and the steps of comparing the determined movementpattern with movement patterns stored in a database, deriving ordetermining of correction parameters, such as at least one movementdirection, for adapting (changing) the movement of the dental treatmentmeans in accordance with the compared movement patterns, and outputtinginformation using an output device in accordance with the derived ordetermined correction parameters.

This method is advantageous, since it outputs feedback or correctioninformation concerning the movement sequence preferably directly to aperson performing a dental treatment, such as a teeth cleaning, orfeedback or correction information concerning the movement sequence canbe output (e.g., in real time) to a person performing the dentaltreatment, should they guide the dental treatment means with a movementthat was assessed, e.g., by software, as being inappropriate or with amovement pattern that was assessed as being inappropriate.

This embodiment is beneficial since e.g. children can be taught (e.g.,supervised) the appropriate movement patterns identified with regard tothe motion sequences during the dental treatment, such as teethcleaning. An optical outputting device, like e.g. a screen of a mobilephone, a watch, a bracelet, a game console or a tablet PC, can output animage or a video that visually displays or outputs to the child thepreferably optimal movement pattern, such as the movement patternsstored electronically in a local storage means and/or on a server and/orin a local or internet based database.

By using an optical detection device or optical capturing device, suchas a camera, which is preferably in signal communication with theoutputting device and which is preferably arranged in the same housingas the outputting device, the motions of the child are detectable andthus can be monitored. In case the child deviates from the displayedmotions or movement patterns, then a signal is preferably output thatclearly shows the deviation to the child. The detection device and theoutputting device, such as for visual and/or haptic and/or acousticoutputting of signals and/or commands, are preferably components of asingle apparatus, such as a mobile apparatus. The apparatus is hereinpreferably configured as a portable processor device, such as a mobilephone or tablet PC.

According to a further embodiment of the present teachings, aregistration of the teeth to be treated preferably takes place, whereinthe registration comprises optical recording (capturing) using thedetection device or a manual recording (inputting) using an input screen(user interface), wherein the number and/or the location and/or theorientation and/or defects of at least individual teeth are recorded,and wherein the derivation or determination of the correction parameterspreferably takes into account the registered teeth. Herein the inputscreen (user interface) can be e.g. visualized by a display means of anend-user device, such as of the end-user device that also comprises thedetection device. Preferably, the number of teeth present and/or whichlocation and/or which orientation and/or which defects that individual,multiple or all teeth have can be specified by a person using an inputdevice, which preferably can be a component of the display means.

As was noted above, the derivation or determination of the correctionparameters preferably takes into account of the registered teeth. Forthis purpose, the registration of the teeth can be carried out e.g.using the detection device, which may be optical such as a camera; i.e.preferably at least one image of the dentition is captured, wherein theimage is analyzed using software and e.g. the number, direction and/orlocation and/or defects of the teeth are captured. Thus, it is alsopossible to capture tooth spaces (gaps) in the dentition.

Additionally or as an alternative, the number, direction and/or locationand/or defects of the teeth may be capturable (inputable) manually, suchas by using an input screen (user interface). Additionally oralternatively, the angle feature and/or sealings and/or dental crownsand/or inlays or filings of a single or of multiple teeth arecapturable, such as by using the optical detection device. The locationof the teeth preferably describes the position of the teeth or whichtooth is the particular tooth (e.g. incisors, canine, premolars,molars). Furthermore, it is conceivable that properties of theindividual teeth or relations between the individual teeth arecapturable.

Gum recession and/or discolorations and/or dental calculus and/or cracksand/or breakages and/or holes and/or dental calculus etc., arepreferably understood as being capturable defects according to thepresent teachings.

Furthermore, a calibration step may preferably take place once,repeatedly or several times. Herein, the determination of concrete(specific) and/or personal properties of the body, such as the height orthe distance of the head to the detection device or to a reference pointor the height or the distance of a part of the head to the detectiondevice or to a reference point, is preferably understood herein as beinga calibration.

Particular preferably, multiple body-fixed points and/or surface pointsare captured by the optical detection device during the calibration,wherein preferably one or multiple anterior teeth and/or the contactregion extending in the tooth length direction or the boundary regionbetween the anterior teeth is captured. Herein, it preferably involvesthe central incisor teeth, such as of the upper jaw, wherein it is alsoconceivable that the central incisor teeth of the lower jaw are also oralternatively captured.

The detection of the position of the incisor teeth of the upper jawand/or the boundary region between those incisor teeth is often possiblewhen the upper lip is pulled up, which however can not always be ensuredduring tooth brushing. Nevertheless, the detection of the position ofthe incisor teeth and/or the boundary region between the incisor teethin relation to further anthropometric points of the face surface, suchas of the nose, cheeks and forehead region, wrinkles, such as foreheadwrinkles, or pupils, provides a possibility for defining a highlyprecise coordinate system and/or reference system on the head of aperson.

For clinical studies, this preferably should or can be referenced withestablished dental coordinate definitions like e.g. the “Frankfurterhorizontal plan”, the “Camperschen plane”, or the occlusal plane.

Furthermore, a coordinate system defined in that way additionally can becoupled to exactly or at least one, two, three further body points or tomultiple body points, such as the nose, the chin, the forehead, one earor two ears, etc. This is beneficial since, on the one hand, the oncedefined coordinate system is always detectable independently of theposition of the lips, whereby the position of the teeth is always known,and on the other hand the shape of the body part, e.g. the nose, isirrelevant. The soft tissue deformation that typically occurs duringmouth opening should or can be compensated by a correction computationto further raise the accuracy of the method.

According to a further embodiment of the present teachings, the opticalrecording (capturing) of an object takes place such that at least theextension of the object in one dimension, i.e. in length and/or in widthand/or in height and/or circumference, is known and the extension of theobject in the known dimension or in the known dimensions is preferablyused for the determination of the dimension parameters, to capturegeometric relations, like e.g. the length of the path travelled by thedental treatment means during a dental treatment and/or the dimension ofthe teeth, such as of the visible parts of the teeth, etc. Herein, theobject is preferably the person who is using the dental treatment means,wherein the known dimension of the person is preferably his/her height.Additionally or alternatively it is conceivable that the object is thedental treatment means, such as the tooth brush, wherein the knowndimension of the toothbrush is preferably the length and/or the widthand/or the height and/or the circumference.

Additionally or alternatively it is conceivable that the object is anelement, the dimensions of which are substantially identical everywhereon earth, like e.g. a CD, DVD or credit card. Preferably, data withrespect to the properties of the dental treatment means are recordableusing an identification means (cf. later embodiments).

In another aspect of the present teachings, a sensor device, such as apressure sensor, for detecting the contact pressure applied by thedental treatment means to the mouth or the teeth and/or the gum may beprovided in or on the dental treatment means or for bringing intocontact with the dental treatment means. Alternatively or additionally,the pressure detection may be performed by the optical detection device.In this case, a bending of the dental treatment means is captured and isevaluated in accordance with known strength values. The strength valuesthus can be stored or are storable in a database.

Furthermore, the dental treatment means may be directly identified orthe packaging thereof (e.g. a bar code or QR-code) may be opticallyrecorded (captured) and identified. The identification of the dentaltreatment means enables thereby, that the information belonging to thespecific dental treatment means regarding its properties, such as thestrength values, are automatically stored for a processor device or areretrieved. Furthermore, the information concerning the properties of thedental treatment means also may be made available or recordable orregisterable manually in a database, such as of a mobile end-userdevice. Preferable the information concerning the properties arestorable electronically either locally on the mobile end-user device oron a server.

Thus, according to a further embodiment of the present teachings, adetermination of the pressure during tooth cleaning, such as toothbrushing, applied by a dental treatment means, such as a toothbrush, tothe teeth or gum of the person performing the tooth cleaning takesplace, wherein an optical detection of the dental treatment means, suchas directly or indirectly, takes place using the optical detectiondevice, wherein information are captured or image data is generated bythe detection device image and a deformation, such as an elasticdeformation, e.g., bending, of the dental treatment means is determinedusing the image information or the image data, and wherein the pressureis determined at least in accordance with the deformation of the dentaltreatment means and preferably in accordance with further data thatcontains properties of the dental treatment means.

Furthermore, a signal or information may be output via an output devicein accordance with the determined pressure. Preferably the signal orinformation represents a guidance for the user of the dental treatmentmeans, such as the person undergoing the dental treatment. Thedetermination of the pressure or the output of the signals and/orinformation to be output in accordance with the pressure preferablytakes place in real time. The output of the signals and/or informationparticular preferably takes place using an optical outputting device.The optical outputting device is preferably a component of the device,such as of the end-user device, to which the optical detection devicealso belongs. The signals and/or information concerning the pressure areparticular preferably communicated simultaneously and/or via the sameoutputting device to the same person; the correction parameters foradapting the dental treatment motions are also thereby communicated to aperson, such as the person undergoing and/or performing the dentaltreatment. The data concerning the properties of the dental treatmentmeans can include material properties and/or strength values and/ordimensions and/or bending stiffness and/or brand and/or age and/or thenumber of usages, such as operating life, etc.

According to a further embodiment of the present teachings, at leastthree preferably predetermined points, such as surface points, of abody, such as of a head, of a person, such as with respect to whichsurface points the movement patterns are recorded, preferably can berecorded using the optical detection device, wherein preferably at leasttwo points are connectable or are virtually connectable or imaginarilyconnectable with one line and at least a third point does not lie onsaid line, wherein the reference system is defined by the at least threerecorded points and wherein movement of the dental treatment means isrecorded with respect to this reference system or wherein at least threepreferably predetermined points of the dental treatment means, such as atooth brush, are recorded using the optical detection device, wherein atleast two points are connectable with one line and at least a thirdpoint does not lie on said line, wherein the reference system is definedby the at least three recorded points and wherein a movement of theperson, such as of the head, is recorded (captured) with respect to thisfurther reference system.

According to a further embodiment of the present teachings, inaccordance with a movement of the body, such as of the head, of theperson with respect to which the movement patterns are recorded,different points of the body (e.g., of the head) of the person arerecorded relative to the optical detection device for the definition ofthe reference system. This embodiment is beneficial since the head ofthe person does not have to be completely motionless during execution ofthe method, but can preferably move and nevertheless a precise detectionof the movement pattern is possible. Should a surface point captured fordefining the reference system not be capturable any more e.g. as aresult of a head rotation by the person, another body fixed or headfixed point or a surface point or multiple body fixed or head fixedpoints or surface points are preferably captured already before saidsurface point can not be captured anymore, by means of which point(s)the reference system can be further maintained. For example, in additionto characteristic shapes, like e.g. the tip of the nose, color changesof the skin, such as a pigment disorder, pigmented nevus, scars, bodydecorations, such as tattoos, the cavity of the eye, the incisor teeth,such as of the upper jaw, as well as points of devices attached to thebody, such as to the head, like e.g. clothes, caps, glasses, can becaptured as the body fixed or head fixed points or the surface points.

According to a further embodiment of the present teachings, theprocessor device is a mobile end-user device (e.g. a mobile phone ortablet PC) that sends the recorded data via the internet to a serverdevice for further processing and/or receives data via the internet foroutput using an outputting means, wherein the data comprises informationconcerning motion parameters of the dental treatment means.

Furthermore, a system for an at least partial determination or an atleast partially indirect determination of a personal and preferablydynamic movement pattern of a dental treatment means (e.g., inparticular a tooth brush), during a dental treatment is disclosed.Herein, a dynamic movement pattern is preferably understood as being amovement pattern, which is at least partially defined by velocityinformation and/or acceleration information of the dental treatmentmeans with respect to a treatment area. The system preferably comprisesat least one detection device, such as a sensor device or an opticaldetection device, for recording (capturing) of data with respect to atleast one movement parameter, such as the movement direction, theacceleration, the traveled path and/or the velocity, of the dentaltreatment means and at least or exactly one data processing device forgenerating movement patterns based on the recorded data.

Herein, the data processing device is preferably a device that is atleast temporarily connectable with the internet, such as a computer, adigital camera, a gaming console, a tablet PC, a laptop, a wristwatch(e.g. a smart watch), a television and/or a mobile phone. Furthermore,the data processing device may generate the movement patterns partiallyor fully.

Particular preferably, the data processing device transmits the fully orpartially generated movement patterns to a server device, whichpreferably receives personal movement patterns of dental treatment meansfrom a plurality of data processing devices. The data transmission cantake place e.g. in accordance with an energy-, connection-, time-, datavolume- and/or dental-treatment-repeating-criteria, wherein thedental-treatment-repeating-criteria preferably specify a specific numberof dental treatments. The connection criteria preferably specify thetype and/or quality of the internet connection, e.g. whether it is amobile phone rate plan or a local network, like e.g. a WLAN. The energycriteria preferably specify the state of charge of an end-user devisepowered, such as with electricity, by a battery.

This criteria dependent transmission of the partially or fully generatedmovement patterns is beneficial, since the computation of the movementpatterns primarily can take place on the side of the data processingdevice, i.e. on the end-user device side, and thus the amount of datatransfer between the individual end-user devices and the server devicecan be kept relatively small. This is further beneficial since theserver device needs to have a significant smaller computing power incase it receives partially or fully prepared movement patterns and canprocess them. The server device preferably does not have to start witheffecting or computing the complete generation of movement patterns fromthe raw data.

However, it is alternatively also conceivable that the data recorded bythe detection device are transmitted to the server device forprocessing. Thereby, transmitting can take place immediately after therecording or in accordance with specific criteria. Herein, the criteriacan be e.g. an energy-, connection-, time-, data volume-, and/ordental-treatment-repeating -criteria. The server device preferably canbe considered in this situation as a data processing device thatprocesses the recorded data; that means a data processing device is notnecessarily required on the end-user side or in the area of thedetection device. Nevertheless, a processor device, such as a mobilephone, is preferably provided that at least effects the transmission ofdata to the server device.

In a particular preferred embodiment, the server device has access toclinically gathered information with respect to the dental healthstatus, the number of dental visits per year and/or the treatmenthistory, the used dental treatment means, etc. of specific persons andconnects this information with the recorded movement pattern of theparticular person using that device. The recorded movement patterns of aspecific person are preferably associated with the specific person by apersonalization instance, like e.g. a name request, a password request,a phone number, an email address or a pseudonym. This embodiment isbeneficial, since unique monitoring and supervision are performableusing the data generated by the server device, which data include thedental treatment status, the number of dental visits per year and/or thetreatment history and the recorded movement parameters or the movementpatterns, and teachings for further improvement of the dental treatmenttechnique are derivable.

A dental treatment means (e.g., a toothbrush, oral douche, gastransmitter, such as an air emitter, sand emitter, driller, ultrasoniccleaner, etc.) that preferably can be employed or utilized in a systemaccording to the present teachings preferably comprises at least onephysical structure, wherein a treatment device for at least indirectbringing into contact with a tooth surface and a contact region forholding the dental treatment means is arranged or attached on thephysical structure, wherein a detection device, such as a position-,velocity-, acceleration- and/or rotation sensor, for detection ofmovement parameter(s) is, at least during usage of the dental treatmentmeans, physically connected with the physical structure. Herein, it isconceivable that the detection device is coupled or coupleable with anydental treatment means. Furthermore, it is conceivable that thedetection device is fixedly or permanently coupled with the electrictoothbrush.

According to a further embodiment of the present teachings, the dentaltreatment means comprises a communication interface for transmitting therecorded data, such as in a raw state or in a processed state, to a dataprocessing device, wherein the communication interface is particularpreferably designed in such a way that the data is transmittablewirelessly.

According to a further embodiment of the present teachings, the systemcomprises a processor device, such as a mobile end-user device, likee.g. a mobile phone, that preferably comprises at least one opticaldetection device, such as a camera, for detecting (recording) data of atleast one motion parameter, such as the movement direction or—rotation,the acceleration, the traveled path and/or the velocity of a dentaltreatment means, and a data processing device for image separation ofthe dental treatment means from at least one other image componentrecorded (captured) by the optical detection device and for thedetermination of at least one movement pattern corresponding to themotion parameter(s).

The optical detection device is preferably used in the system as adetection device for the at least partial determination of a personalmovement pattern during a dental treatment, such as a teeth cleaning.

In case the detection device is preferably designed as an opticaldetection device or optical capturing (recording) device, the dentaltreatment means may have an optically-distinctive shape, whichpreferably corresponds to an axis of an abscissa or serves to furtherfacilitate a three-dimensional recording of one specific motionparameter of the dental treatment means during usage of exactly or atleast one camera.

However, additionally or alternatively the dental treatment means alsomay be equipped with one or multiple sensor devices. In this case, thedata processing device uses the sensor data and the data of the opticaldetection device for the determination of the motion parameter.

Furthermore, the dental treatment means also may be equipped with one ormultiple sensor devices, in case the processor device or the systemcomprises at least or exactly one, two or multiple cameras, such as 3cameras. In this case the data processing device can utilize data ofone, some or all of the cameras and/or of the sensors for thedetermination of the motion parameter.

According to a further embodiment of the present teachings, the opticaldetection device comprises at least one, such as exactly or at leasttwo, cameras for three-dimensional recording (capturing) of the at leastone motion parameter of a dental treatment means, wherein the processordevice defines a three-dimensional coordinate system or reference systemthat moves with the dental treatment means together in space duringmovement of the dental treatment means. Hereby the coordinate- orreference system can be constituted (defined) by characteristicanthropometric body points, body lines and/or body surfaces of a bodypart of a person, such as the hand with which the dental treatment meansis being guided. However it also can be constituted additionally bysurface points, surface lines and/or surface areas of the dentaltreatment means. The second camera is preferably fixedly arranged inrelation the first camera, so that a once-only calibration of thedetection system can take place before utilization. It is preferablyalso conceivable that the at least two cameras are moveable with respectto each other. The at least two cameras are preferably at least, exactlyor maximally 2, 3, 4, 5 or 6 cameras.

According to a further embodiment of the present teachings, the opticaldetection device records (detects) a part (portion) of the head or atleast three points of the surface of the head or three body-fixedpoints, which move during a movement of the person, such as of the head,together with the person, such as with the head, and define (establish)a three-dimensional coordinate system or reference system in therecorded part of the head or by means of these points, wherein amovement of the dental treatment means is determined in accordance withthe defined coordinate system of the head or a movement of the recordedpart of the head is determined in accordance with the coordinate systemof the dental treatment means. Herein it is conceivable that thecoordinate system is directly associated with the dental treatmentmeans, wherein it is additionally or alternatively also conceivable thatthe coordinate system is at least partially (preferably completely)defined by defined body surface parts, such as two or more knuckles ofthe hand constituting the linkage between the palm of the hand and thefingers.

This embodiment is beneficial since: (i) a movement of any dentaltreatment means, such as any customary toothbrush or electrictoothbrush, is optically at least indirectly at least partiallydetectable at least by recording (detecting) the hand movement and (ii)optical detection devices, such as hand cameras, are very wide spread,whereby the invention is applicable for an extremely large number ofpersons.

Particular preferably the dental treatment means is identifiable andseparable from the remaining image parts by the data processing devicein such a way that its movements, in particular with respect to orrelative to a reference or coordinate system fixed on the head or in theregion of the head of the person, are recordable and particularpreferable analyzable.

Furthermore, preferably at least or exactly one body part of a person isadditionally identifiable and separable from the remaining image partsby the data processing device in such a manner that a motion parameterof the dental treatment means is determinable or computable with respectto the body part. The body part is preferably a part (portion) of thehead, like e.g. forehead, eyebrows, eye hole, mouth, lips, ears, cheeksand/or chin; more preferably, it is one or multiple body-fixed, such ascranium fixed, parts, like e.g. forehead, chin and/or nose.

However, it is also conceivable that multiple body parts, like e.g. theeyes, the mouth and/or nose are recorded. It is thus conceivable thatone or multiple photos of the user or the patient, such as from the heador the face of the user, are recorded and stored.

Individual body shapes, such as face-part-shapes or shapes ofcharacteristic face features, are preferably manually or automaticallyrecordable or definable or are registerable as reference points orreference surfaces, with respect to which one or multiple motionparameters of the dental treatment means are recorded. For example, oneor two eyes, the nose, the mouth, one or two ears and/or one or two lipscan be used or registered as the reference point(s).

Furthermore, additionally or alternatively, the eye brows or the eyes,such as the pupils, of a person can be recorded. One line or axis can bedefined or set connecting the eyes or the pupils, such as the centers ofthe eyes or pupils. This line or axis can define a direction, such asthe x-, y- or z-direction, of a coordinate system.

Furthermore, a second line or axis can be defined or set in thedirection of extension of the nose, such as of the nasal bridge, of thatperson which preferably defines a second direction differing from thefirst direction, such as the x-, y- or z-direction, of the coordinatesystem. Preferably the first direction and the second direction aresubstantially or exactly oriented orthogonally to each other. A thirddirection preferably extends orthogonally with respect to a planedefined by the first and second direction.

The recording of a movement of the dental treatment means with respectto a coordinate system coupled with a movement of the head is extremelybeneficial since the person carrying out the dental treatment can movehis/her head and/or body without causing the recording (capturing) ofthe movement patterns to become unprecise or impossible.

Furthermore, the present teachings concern the usage of a position-,velocity-and/or acceleration sensor or a video tracking system for thedetermination of a movement pattern of a dental treatment means during adental treatment.

In another aspect of the present teachings, a method is disclosed for atleast partially determining of a pressure resulting during a dentaltreatment, such as a teeth cleaning, and applied by a dental treatmentmeans to the teeth or the gum of a person undergoing the dentaltreatment. The method preferably comprises:

(i) Performing an at least indirect and preferably direct opticaldetection of at least one part of the dental treatment means during thedental treatment using an optical detection device or performing anoptical recording (detection) of the dental treatment means during thedental treatment using an optical detection device. Herein, it ispreferably conceivable that, in the case of an indirect recording of thedental treatment means, a position and/or orientation and/or alignmentof the hand, which is guiding the dental treatment means, is recorded.An indirect recording of the dental treatment means preferably describesthe real (actual) optical recording of the dental treatment means.

(ii) Generating image data representing the optical recording, whereinthe image data preferably comprises or represents several grades ofdeformation of the dental treatment means. Preferably a bending and/ortorsion of a toothbrush shaft and/or of a toothbrush head is recorded asa deformation. However it is additionally or alternatively alsoconceivable that a deformation of the bristles of a toothbrush isrecorded or determined.

(iii) Analyzing the image data to determine the particular degree ofdeformation of the dental treatment means.

(iv) Determining the pressure, wherein the pressure is determined atleast in accordance with the deformation of the dental treatment meansand in accordance with further dental treatment means data, wherein thedental treatment means data comprises data concerning properties of thedental treatment means.

Furthermore, a signal or information may be output via an output devicein accordance with the determined pressure. Preferably the signal or theinformation represents a guidance for the user of the dental treatmentmeans, such as for the person undergoing the dental treatment. Therecording (detection) of the pressure or the output of the signals to beoutput in accordance with the detected pressure preferably takes placein real time.

The signals and/or information are, particular preferably, output usingan optical outputting device. The optical outputting device ispreferably a component of a device to which also the optical detectiondevice belongs. The signals and/or information with respect to thepressure are particular preferably simultaneously and/or using the sameoutputting device communicated to the same person, which is also used tocommunicate correction parameters for adapting the dental treatmentmovements of a person, such as the person who is undergoing and/orperforming the dental treatment.

The data concerning the properties of the dental treatment means caninclude material properties and/or strength values and/or dimensionsand/or bending stiffness and/or brand and/or age and/or number of usage,such as duration of usage, etc.

According to a further embodiment of the present teachings, an object isoptically recorded (captured) such that at least the extension of theobject in one dimension, i.e. the length and/or width and/or heightand/or circumference, is known, and the extension(s) of the object inthe known dimension(s) is (are) used for the determination of dimensionparameters in order to detect geometric relationships, like e.g. thelength of the path traveled by the dental treatment means during thedental treatment and/or the dimensions of the teeth, such as of thevisible parts of the teeth and/or the dimensions of the dental treatmentmeans and/or the distance of the teeth and/or the dental treatment meanswith respect to the detection device, etc. Herein, the object ispreferably the person using the dental treatment means, wherein theknown dimension of the person preferably is his/her height. It isadditionally or alternatively conceivable that the object is the dentaltreatment means, such as the toothbrush, wherein the known dimension(s)of the toothbrush is (are) preferably the length and/or width and/orheight and/or circumference.

Furthermore, the present teachings also relate to a system or a methodcomprising:

-   -   at least an optical detection device, such as at least one        camera, for recording (capturing) of data with respect to at        least one motion parameter, such as the movement direction, the        acceleration, the travelled path and/or the velocity, of a        dental treatment means and    -   a data processing device for separation of an image part        representing the dental treatment means from at least one other        image part recorded by the optical detection device and for        determining at least one movement pattern corresponding to the        motion parameter using the separated image part,    -   wherein the optical detection device comprises preferably at        least one camera, such as exactly or at least two cameras, for        the preferably three-dimensional recording (capturing) of the at        least one motion parameter of a dental treatment means,    -   wherein the processor device preferably defines a        three-dimensional coordinate system moving together in space        with the hand guiding the dental treatment means during movement        of the dental treatment means,    -   wherein the optical detection device preferably records a part        (portion) of the head of a person and defines a        three-dimensional coordinate system on the recorded part of the        head,    -   wherein movement of the dental treatment means is preferably        determined in accordance with the defined coordinate system of        the head or wherein movement of the recorded part (portion) of        the head is determined in accordance with the coordinate system        of the dental treatment means,    -   wherein the processor device is preferably a mobile phone or a        tablet PC and sends the recorded data preferably via the        internet to a server device for further processing and/or        receives via internet data for outputting by an outputting        device of the mobile phone or the tablet PC, and    -   wherein the data preferably comprise information concerning        motion parameters of the dental treatment means and/or of the        pressure, with which the dental treatment means is pressed        against the teeth and/or the gum.

A database is preferably provided that comprisesdental-treatment-means-data. Herein, it is conceivable that the databaseis provided locally on the end-user device and/or on a server. Anidentification means, such as a software based identification means, ispreferably provided. The identification means records (captures), e.g.optically captures, the dental treatment means or information thatrepresents the dental treatment means, like e.g. a QR-code. Theoptically recorded (captured) dental treatment means is preferablyrepresented by image information or image data, which are preferablycorrelated with image information or image data with respect to aplurality of dental treatment means provided in said database. Data orinformation concerning the properties of the individual dental treatmentmeans are preferably stored in the database. In case of a match of thedata or information contained in the database with the recorded imagedata or image information, preferably a selection and utilization of thedata or information stored in the database with respect to the specificor identified dental treatment means takes place to determine thepressure.

A link with the data or information, which are provided in the database,associated with a dental treatment means is further possible using alinkage, like e.g. by reading out a code, such as an optical code, likee.g. a bar code or QR-code. Furthermore, it is additionally oralternatively conceivable that a screen (user interface) for manuallyinputting of the dental-treatment-means-data or—properties is providedand the manually inputted data or information are used for determiningthe pressure. Additionally or alternatively, it is conceivable that thedata or information concerning the dental treatment means aretransferred into the database using a radio device, like e.g. anear-field-communication (NFC) or Bluetooth, or a linkage to thedatabase is generated or the data for determining the pressure arestored.

The present teachings further relate to a computer program product forexecuting of one or multiple above- or below-described methods and/orfor statistical analysis of the generated datasets.

The terms “optical detection device” and “optical capturing device” canbe understood as being synonymous in the context of the presentdisclosure.

Further benefits, goals and features of the present teachings will beunderstood upon reading the following description of the attachedfigures, in which exemplary features according to the present teachings,such as processor devices or systems for recording the movement patternsduring a dental treatment, are illustrated. Components of the devicesand methods according to the present teaching, which match at leastsubstantially with respect to their function can be marked with the samereference symbol, wherein such components do not have to be marked ordescribed in all figures.

BRIEF DESCRIPTION OF THE DRAWINGS

In the following, embodiments of the invention are described in anexemplarily manner with reference to the attached figures.

FIG. 1A shows one embodiment of a system according to the presentteachings for recording (capturing) dental treatment movements;

FIG. 1B shows an internet based network according to the presentteachings, in which multiple systems are involved;

FIG. 2B shows an outstretched hand with characteristic surface points orsurface parts; and

FIG. 2B shows a hand clenched into a fist with characteristic surfacepoints or surface parts, wherein the hand is holding a dental treatmentmeans (toothbrush).

DETAILED DESCRIPTION OF THE INVENTION

In FIG. 1A, a system 1 for recording (capturing) dental treatmentmovements using a processor device 2 or a mobile end-user device 2 isshown. The mobile end-user device 2, which is preferably configured as amobile phone, herein preferably has at least or exactly one opticaldetection device 4, such as an optical capturing device 4, that can bepreferably formed as a camera. The camera preferably records (captures)exactly or more than 15 fps (frames per second), exactly, up to or morethan 30 fps, exactly, up to or more than 45 fps, exactly, up to or morethan 60 fps, exactly, up to or more than 75 fps, exactly, up to or morethan 90 fps, exactly, up to or more than 120 fps or exactly, up to ormore than 200 fps.

Reference number 6 denotes the recording (detection) area, in whichpreferably at least one part (portion) of the head, such as one or bothpupils, the nose and/or the mouth, of a person 8 as well as at least onepart (portion) of the hand 5 guiding the dental treatment means 14 arepresent. Reference numbers 10 and 12 denote preferably person-fixed axesor coordinate directions, which follow a movement of the head preferablyexactly. Reference number 14 denotes a dental treatment means that isformed as toothbrush, which is moved in space according to referencenumber 16 and is thus moved relative to the head.

Motion sequences for correcting the movement and/or for specifying amotion sequence are displayable on the optical outputting device(display, screen) 18 of the processor device 2.

FIG. 1B shows that multiple processor devices 2 transmit via a datalink, such as an internet link, data to a server that preferablygenerates optimized movement characteristics, such as in accordance withmedical condition histories or defect histories of the individualpersons.

Both FIGS. 1A and 1B underlie a method according to the presentteachings for determining a movement pattern of a dental treatmentmeans, such as a toothbrush, during a dental treatment, such as a teethcleaning. Such as method preferably comprises the following steps 1 and2:

1. Moving the dental treatment means to treat surface parts of the teethin at least an X-/Y-plane, wherein data concerning at least one motionparameter (e.g., the movement direction or—rotation, the acceleration,the path and/or the velocity) of the dental treatment means 14 relativeto a reference system 10 that moves with the head 8 of the treatedperson are recorded (detected) by the at least one optical detectiondevice 4. The data preferably consist of image information, like e.g. anarrangement of pixels, and preferably describe (represent) at least onepart (portion) of the head 8 of the person and at least one part of thehand 5 of the person, with which the person is guiding the dentaltreatment means. Reference number 7 denotes the arm of the person fromwhich the hand 5 guiding the dental treatment means 14 extends.

The processor device 2 defines a further three-dimensional referencesystem 16, that moves in space together with the dental treatment means14 during a movement of the dental treatment means 14, wherein thefurther reference system 16 is particular preferably defined bycharacteristic anthropometric body points of the hand, body lines of thehand and/or body surfaces of the hand, with which the dental treatmentmeans 14 is guided.

This further three-dimensional reference system 16 defined by handparts/points/features is beneficial since the dental treatment means 14is typically partially or even completely enclosed by the hand 5 of theuser and thus can be at least partially invisible (hidden) for theoptical detection device 4. Based on the movement of the handcharacteristic(s), such as anthropometric, body points of the hand, bodylines of the hand and/or body surfaces of the hand defining the furtherreference system 16, the position and/or orientation of the dentaltreatment means 14 is determinable. The processor device 2 preferablyanalyzes data, such as image information, recorded (captured) by theoptical detection device 4 to define the further three-dimensionalreference system 16.

However, it is also conceivable that the data recorded by the opticaldetection device 4 are processed or analyzed by a processor device 2,which is located outside the mobile end-user device. Herein, theprocessor device may be a server device that receives the data via aninternet connection. The processor device configured as a server devicepreferably sends the processed or modified or analyzed data or databased thereon via an internet connection to the end-user device.

2. Providing the recorded (captured) data to the processor device or toa further processor device for the determination of the movement patternas well as preferably the determination of the movement pattern usingthe processor device or using the further processor device. Herein, itis conceivable the further reference system is defined e.g. using aprocessor device of a mobile or stationary end-user device. Datarecorded by the optical detection device 4 with respect to the referencesystem 10 moving with the head 8 of the person and with the respect tothe reference system 16 moving with the hand 5 of the person ispreferably analyzed. Particular preferably the relative movements of thereference systems 10, 16 are determined with respect to each other.

Furthermore, the orientation and/or position of the dental treatmentmeans 14, such as of the bristle part of the toothbrush, is determinedwith respect to the hand 5 guiding the dental treatment means 14preferably at least temporarily, such as before and/or during toothbrushing. The determination of the position and/or orientation of thedental treatment means 14, such as of the bristle part of thetoothbrush, preferably takes place by analyzing the data determined bythe optical detection device 4.

FIG. 2B shows the back of the hand 5, wherein reference numbers 48, 49,50 and 51 denote purely exemplary characteristic points on the surfaceof the hand. Thus, reference number 48 denotes the transition betweentwo fingers. Reference numbers 49 and 50 denote the knuckles of the handand reference number 51 denotes a knuckle of a finger. It has beenrecognized that these body surface points or -parts and similar bodysurface points or—parts are always very characteristic and are thereforevery suitable for an optical recording (detection), whereby the presentteachings facilitate a very simple and functionally very reliable methodfor a user.

In FIG. 2B the hand 5 is shown in a purely exemplary manner togetherwith a dental treatment means 14. This illustration shows individualcharacteristic body points or—sections, as they are preferably recorded(detected) by the optical detection device 4.

1-10.(canceled)
 11. A method comprising: selecting one treatment modefrom a plurality of treatment modes displayed on a screen of a mobilephone or tablet PC, using a camera of the tablet PC or mobile phone tocapture an image area containing (i) one or both pupils, the nose,eyebrows, eye hole, lips, ears, cheeks and/or chin of the person, and(ii) a toothbrush moving at least in an X-/Y-plane to treat surfaceparts of teeth of the person, wirelessly receiving sensor data from thetoothbrush in the mobile phone or tablet PC, the sensor data containingat least rotational movements and/or acceleration of the toothbrushcaptured by an acceleration sensor and/or a rotation sensor, disposed inor on the toothbrush, processing the sensor data in a processor, therebygenerating processed sensor data, generating correction informationbased on the processed sensor data and the selected treatment mode, anddisplaying the correction information on the screen of the mobile phoneor tablet PC.
 12. The method according to claim 11, further comprising:detecting a contact pressure between the toothbrush and the teeth usinga pressure sensor associated with the toothbrush.
 13. The methodaccording to claim 12, further comprising: haptically or acousticallyoutputting signals or commands from the toothbrush in accordance withthe correction information.
 14. The method according to claim 13,further comprising: transmitting raw or processed movement pattern datafrom the mobile phone or tablet PC to a server, and sending processed,modified or analyzed data via an internet connection from the serverback to the mobile phone or tablet PC.
 15. A method comprising:selecting one treatment mode from a plurality of treatment modesdisplayed on a screen of a mobile phone or tablet PC, using a camera ofthe tablet PC or mobile phone to capture an image area containing (i) atleast one body part selected from one or both pupils, the nose,eyebrows, eye hole, lips, ears, cheeks and/or chin of the person, aswell as (ii) a toothbrush moving at least in an X-/Y-plane to treatsurface parts of teeth of the person, thereby generating image data,wirelessly receiving sensor data from the toothbrush in the mobile phoneor tablet PC, the sensor data containing at least rotational movementsand/or acceleration of the toothbrush captured by an acceleration sensorand/or a rotation sensor, disposed in or on the toothbrush, processingthe sensor data in a processor, thereby generating processed sensordata, using the processor of the mobile phone or the tablet PC toidentify the toothbrush in the image data based upon the shape of thetoothbrush and to separate toothbrush image data from said image data insuch a way that movements of the toothbrush are analyzable based uponthe separated toothbrush image data, using the processor to identify theat least one body part of the person in the image data and to separatebody part image data from the remaining image parts of said image data,using the processor to determine a motion parameter of the toothbrushidentified in the separated toothbrush image data with respect to thebody part identified in the separated body part image data, generatingcorrection information based on the processed sensor data and the motionparameter of the toothbrush and the selected treatment mode, anddisplaying the correction information on the screen of the mobile phoneor tablet PC.
 16. The method according to claim 15, further comprising:performing a registration of the teeth to be treated, thereby generatingregistered teeth data, wherein the determination of the correctionparameters is also based in part on the registered teeth data.
 17. Themethod according to claim 16, further comprising: haptically andvisually outputting signals or commands from the toothbrush inaccordance with the correction information.
 18. The method according toclaim 17, further comprising: transmitting raw or processed movementpattern data from the mobile phone or tablet PC to a server, and sendingprocessed, modified or analyzed data via an internet connection from theserver to the mobile phone or tablet PC.
 19. The method according toclaim 15, further comprising: detecting a contact pressure between thetoothbrush and the teeth using a pressure sensor associated with thetoothbrush, generating in real time a guidance for the person using thetoothbrush based upon the detected contact pressure, and visuallydisplaying the guidance regarding the detected contact pressure on thescreen of the mobile phone or tablet PC.
 20. The method according toclaim 19, further comprising: visually displaying an animatedrepresentation of the toothbrush image data and/or the body part data onthe screen of the mobile phone or tablet PC, wherein the animatedrepresentation is generated based upon the correction parameters.
 21. Anon-transitory computer readable medium storing a program causing one ormore processors to execute the method according to claim
 11. 22. Anon-transitory computer readable medium storing a program causing one ormore processors to execute the method according to claim
 15. 23. Amobile phone or tablet PC comprising: a processor, a display, and thenon-transitory computer readable medium according to claim
 22. 24. Amethod comprising: selecting one treatment mode from a plurality oftreatment modes displayed on a screen of a mobile phone or tablet PC,using at least an acceleration sensor or a rotation sensor that are apart of a toothbrush to detect movement data, wherein the accelerationsensor and/or the rotation sensor sense rotational movements and/oracceleration of the toothbrush while the toothbrush is moving at leastin an X-/Y-plane to treat surface parts of the teeth of a person,thereby generating motion sensor data, wherein the toothbrush comprisesa brush head for contacting a tooth surface, a handle for holding thetoothbrush, and at least one outputting device configured to visuallyand/or haptically output signals or commands, detecting a contactpressure between the brush head and the teeth using a pressure sensorassociated with the toothbrush, thereby generating contact pressuredata, wirelessly communicating the motion sensor data and the contactpressure data via a communication interface of the toothbrush to amobile phone or tablet PC, processing the motion sensor data and thecontact pressure data in a processor, generating in real time atoothbrush movement correction guidance based on the processed motionsensor data and the selected treatment mode, generating in real time acontact pressure guidance based upon the processed contact pressure dataand the selected treatment mode, and visually displaying (i) thetoothbrush movement correction guidance and/or (ii) the contact pressureguidance on a screen of the mobile phone or tablet PC.
 25. The methodaccording to claim 24, further comprising: assigning the movementpattern data to a specific phone number or email address.
 26. The methodaccording to claim 25, further comprising: performing a registration ofthe person's teeth by manually inputting information concerning theteeth using an input screen visualized on the screen of the mobile phoneor tablet PC, thereby generating registered teeth data, wherein thedetermination of the toothbrush movement correction parameters is basedin part on the registered teeth data.
 27. The method according to claim25, further comprising: using a camera of the tablet PC or mobile phoneto capture more than 15 frames per second of an image area containingone or both pupils, the nose, eyebrows, eye hole, lips, ears, cheeksand/or chin of the person, thereby generating image data, and displayingan animated representation of the captured image area on the screen ofthe mobile phone or tablet PC.
 28. A non-transitory computer readablemedium storing a program causing one or more processors to execute themethod according to claim
 24. 29. A mobile phone or tablet PCcomprising: a processor, a display, and the non-transitory computerreadable medium according to claim
 21. 30. A mobile phone or tablet PCcomprising: a processor, a display, and the non-transitory computerreadable medium according to claim 28.